add patch for adding a write memory barrier to all OpenMPI 4.1.x easyconfigs by bedroge · Pull Request #19940 · easybuilders/easybuild-easyconfigs

bedroge · 2024-02-20T10:53:41Z

This solves a bug in the smcuda btl that causes MPI applications to crash or hang on Neoverse V1 CPUs, see open-mpi/ompi#12270. The issue was fixed in open-mpi/ompi#12344 for OpenMPI 4.1.x (4.1.7 should include the fix).

bedroge · 2024-02-20T10:56:21Z

@boegelbot please test @ generoso

boegelbot · 2024-02-20T11:00:09Z

@bedroge: Request for testing this PR well received on login1

PR test command 'EB_PR=19940 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs /opt/software/slurm/bin/sbatch --job-name test_PR_19940 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 12926

Test results coming soon (I hope)...

Details

- notification for comment with ID 1953959787 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

bedroge · 2024-02-20T12:36:26Z

@boegelbot please test @ jsc-zen3

boegelbot · 2024-02-20T12:40:07Z

@bedroge: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=19940 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_19940 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

exit code: 0
output:

Submitted batch job 3642

Test results coming soon (I hope)...

Details

- notification for comment with ID 1954126426 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2024-02-20T15:26:15Z

Test report by @boegelbot
SUCCESS
Build succeeded for 14 out of 14 (14 easyconfigs in total)
cns1 - Linux Rocky Linux 8.9, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/boegelbot/ef2751a131a3499141f02d45a7eb29aa for a full test report.

boegelbot · 2024-02-20T16:21:55Z

Test report by @boegelbot
FAILED
Build succeeded for 15 out of 16 (14 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/7b076361a2cf4873b7486a6b07a4ed4a for a full test report.

bedroge · 2024-02-21T09:03:56Z

Test report by @boegelbot
FAILED
Build succeeded for 15 out of 16 (14 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/7b076361a2cf4873b7486a6b07a4ed4a for a full test report.

They all succeeded except OpenMPI-4.1.1-intel-compilers-2021.4.0.eb:

============================================================================
== Configuring Open MPI
============================================================================

*** Startup tests
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for x86_64-pc-linux-gnu-gcc... icc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... configure: error: in `/tmp/boegelbot/OpenMPI/4.1.1/intel-compilers-2021.4.0/openmpi-4.1.1':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
 (at easybuild/easybuild-framework/easybuild/tools/run.py:682 in parse_cmd_output)

Not sure what's wrong here...

boegel · 2024-02-21T15:48:02Z

Test report by @boegel
SUCCESS
Build succeeded for 14 out of 14 (14 easyconfigs in total)
node3106.skitty.os - Linux RHEL 8.8, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz, Python 3.6.8
See https://gist.github.com/boegel/fb69197b37e2604f56a805224de670e5 for a full test report.

boegel · 2024-02-21T19:16:22Z

Test report by @boegelbot
FAILED
Build succeeded for 15 out of 16 (14 easyconfigs in total)
jsczen3c1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.3, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.9.18
See https://gist.github.com/boegelbot/7b076361a2cf4873b7486a6b07a4ed4a for a full test report.

They all succeeded except OpenMPI-4.1.1-intel-compilers-2021.4.0.eb:
============================================================================
== Configuring Open MPI
============================================================================

*** Startup tests
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for x86_64-pc-linux-gnu-gcc... icc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... configure: error: in `/tmp/boegelbot/OpenMPI/4.1.1/intel-compilers-2021.4.0/openmpi-4.1.1':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
 (at easybuild/easybuild-framework/easybuild/tools/run.py:682 in parse_cmd_output)
Not sure what's wrong here...

It's related to RHEL9 and the newer glibc:

In file included from conftest.c(10):
/usr/include/stdio.h(824): error: attribute "__malloc__" does not take arguments
    __attribute_malloc__ __attr_dealloc (pclose, 1) __wur;
                         ^

compilation aborted for conftest.c (code 2)
configure:6725: $? = 2
configure:6732: ./conftest
./configure: line 6734: ./conftest: No such file or directory

It seems like that Intel compilers version is basically not compatible with RHEL 9.

I won't let that block this PR, the problem is not caused by the patch being added.

boegel

lgtm

boegel · 2024-02-21T19:17:22Z

Going in, thanks @bedroge!

bedroge added 2 commits February 20, 2024 11:43

patch based on open-mpi/ompi#12344

34c3912

add OpenMPI-4.1.x_add_atomic_wmb.patch and update checksums accordingly

7d8310d

remove execute permissions on file

1ee886f

bedroge added the bug fix label Feb 20, 2024

This was referenced Feb 20, 2024

{mpi}[GCC/13.3.0] OpenMPI v5.0.3, PMIx v5.0.2 #17561

Merged

{2023.06}[foss/2023a] OpenFOAM v10 EESSI/software-layer#404

Merged

boegel added this to the release after 4.9.0 milestone Feb 20, 2024

boegel approved these changes Feb 21, 2024

View reviewed changes

boegel merged commit 01084d1 into easybuilders:develop Feb 21, 2024

bedroge deleted the openmpi_4.1.x_add_wmb branch February 21, 2024 20:04

bedroge added the EESSI Related to EESSI project label Feb 21, 2024

boegel added the aarch64 Related to Arm 64-bit (aarch64) label Feb 28, 2024

This was referenced Mar 13, 2024

{bio,tools}[foss/2023b] GROMACS v2024.1, networkx v3.2.1 #20102

Merged

{bio}[foss/2023a] GROMACS v2023.4 #19728

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add patch for adding a write memory barrier to all OpenMPI 4.1.x easyconfigs#19940

add patch for adding a write memory barrier to all OpenMPI 4.1.x easyconfigs#19940
boegel merged 3 commits intoeasybuilders:developfrom
bedroge:openmpi_4.1.x_add_wmb

bedroge commented Feb 20, 2024

Uh oh!

bedroge commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

bedroge commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

bedroge commented Feb 21, 2024

Uh oh!

boegel commented Feb 21, 2024

Uh oh!

boegel commented Feb 21, 2024

Uh oh!

boegel left a comment

Uh oh!

boegel commented Feb 21, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

bedroge commented Feb 20, 2024

Uh oh!

bedroge commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

bedroge commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

boegelbot commented Feb 20, 2024

Uh oh!

bedroge commented Feb 21, 2024

Uh oh!

boegel commented Feb 21, 2024

Uh oh!

boegel commented Feb 21, 2024

Uh oh!

boegel left a comment

Choose a reason for hiding this comment

Uh oh!

boegel commented Feb 21, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants